Auditory-Visual Integration of Talker Gender in Cantonese Tone Perception

This study investigated the auditory-visual integration of talker gender in the perception of tone variances. Two experiments were conducted to evaluate how listeners use the information of talker gender to adjust their expectation towards speakers’ pitch range and uncover intended tonal targets in Cantonese tone perception. Results from an audio-only tone identification task showed that tone categorization along the same pitch continuum shifted under different conditions of voice gender. Listeners generally heard a tone of lower pitch when the word was produced by a female voice, while they heard a tone of higher pitch when the word was produced at the same pitch level by a male voice. Results from an audio-visual tone identification task showed that tone categorization along the same pitch continuum shifted under different conditions of face gender, despite the fact that the photos of different genders were disguised for the same set of stimuli in identical voices with identical pitch heights. These findings show that gender normalization plays a role in uncovering linguistic pitch targets, and lend support to a hypothesis according to which listeners make use of socially constructed stereotypes to facilitate their basic phonological categorization in speech perception and processing.